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Abstract 


Problems 


The  two  primary  problems  encountered  in  this  study  were  with  the  Institutional  Review 
Board  (IRB)  approval  and  unexpected  staff  turnover.  This  minimal  risk  proposal  was 
submitted  to  the  MRMC  IRB  for  approval  as  CHPPM  does  not  have  an  IRB.  The 
MRMC  IRB  had  been  used  as  the  primary  IRB  in  the  past  and  it  was  assumed  that  the 
MRMC  IRB  approval  was  all  that  would  be  required;  however,  after  the  proposal  had 
been  reviewed,  corrected  and  re-submitted,  the  PI  was  informed  that  the  proposal  would 
need  to  be  submitted  to  each  of  the  IRBs  for  the  target  bases.  Since  time  was  a  concern,  it 
was  decided  to  re-target  bases  under  one  IRB  rather  than  submit  the  proposal  to  four 
different  IRBs.  The  target  bases  were  changed  to  Fort  Drum,  Fort  Eustis,  Fort  Knox, 
Fort  Lee  and  Fort  Bragg.  All  of  these  bases  fall  under  the  WRAMC  IRB.  The  proposal 
received  WRAMC  IRB  approval  without  any  problem.  Unfortunately,  this  time  delay 
could  have  been  avoided  if  the  PI  had  been  informed  that  the  MRMC  approval  alone 
would  not  suffice. 

The  unexpected  staff  turnover  created  delays  in  the  final  analysis  of  the  data.  All  of  the 
data  had  been  collected  prior  to  the  staff  member’s  departure;  however,  not  all  of  the 
data  had  been  entered  for  analysis  and  only  a  very  few  analyses  had  been  conducted. 
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Deliverables 


The  primary  deliverable  is  the  final  tele-ergonomics  methodology  and  the  revised 
assessment  tools  package.  Based  on  the  study  results,  the  assessment  tools  package  is 
being  revised  to  include  decision  charts  and  other  user-  friendly  features.  The  assessment 
tools  are  being  structured  to  allow  development  of  computer  programs  to  assist  in  the 
local  data  collection.  This  package  addresses  both  injury  prevention  and  the  management 
of  soldier/worker  return  to  work.  Return-to-work  assessment,  planning  and 
accommodation  are  critical  pieces  of  clinical  management  of  injuries  as  these  elements 
reduce  lost  work  time  /  profile  periods  as  well  as  reduce  or  prevent  re-  injury.  Both 
prevention  and  return-to-work  impact  on  health  care  system  utilization  by  reducing  the 
occurrence  and  severity  of  injuries. 

The  success  of  the  tele-ergonomics  methodology  is  primarily  evaluated  by  process 
measures,  including  the  accuracy  and  feasibility  of  the  methodology;  decreased  response 
time  to  evaluate  a  problematic  job,  task,  tool  or  equipment;  increased  access  to  expert 
ergonomist  evaluations;  and  the  quality  of  the  evaluation  and  recommendations. 
Although  decreased  injury  rates  and  decreased  limited  duty/lost  duty  time  could  be 
attributed  to  the  tele-ergonomics  methodology,  many  other  local  Ergonomics  Program 
features  and  initiatives  contribute  to  this  outcome.  The  specific  and  quantifiable 
contribution  of  the  tele-ergonomics  package  to  the  reduction  of  these  outcomes  measures 
is  difficult  to  impossible  to  identify  with  confidence. 

The  true  value  of  the  final  product  is  in  the  distribution  of  the  tele-ergonomics 
methodology  and  revised  tool  package  to  all  Army  installations,  facilities,  depots  and 
activities.  Local  Ergonomics  Programs  are  frequently  understaffed  and  lack  the  expert 
ergonomists’  skills  and  knowledge.  This  package  will  dramatically  increase  local 
programs’  access  to  the  experts  and  enhance  the  local  programs’  functioning, 
responsiveness  and  quality  as  well  as  contribute  to  the  overall  program  goal  of  reducing 
injuries  and  lost  work  time. 

The  package  will  also  be  included  in  the  next  build  of  tbe  Defense  Occupational  and 
Environmental  Health  Readiness  System  (DOEHRS)  and  in  the  Deployment  Surveillance 
initiative.  In  addition,  91S  and  local  technician  training  requirements  and  program  will 
be  submitted  for  inclusion  in  the  School  training,  included  in  91S  continuing  education 
initiatives  and  included  in  the  Army  40-hour  Applied  Ergonomics  Course.  Finally,  the 
package  will  be  considered  for  DOD-wide  distribution  through  the  DOD  Ergonomics 
Working  Group. 


Expenditures 


lJU.lU.UUJ  pULUpUlLJJpillJ  UJ^JiJ 


1 

30  FY 

00 

4Q  FY 

00 

IQFY 

01 

1 

. i 

i| 

Element  of  Resource 
(EOR) 

Apr  1  - 
May  31 

Jun  1  - 
Sep  30 

Octl- 
Dec  31 

Jan  1  -  i 
Mar  31  | 

T01AI.S  1 

I  Travel  2100 

0.00 

35,000.00  i 

0.00 

0.001 

35,000.00 

i  .  . 

Shipping  2200 

0.00 

0.00 

0.00 

0.00  i 

0.00! 

1 

j 

Rent  &  Communications  ^ 
2200 

0.00 

0.00 

0.00 

0.00 

0.00^ 

i 

1  Contract  for  Services 

2500 

0.00 

105,000.00 

0.00 

0.00 

105,000.00 ! 

ii 

Supplies 

2600 

0.00 

1,500.00 ! 

0.00 

0.00  i 

1,500.00 

Equipment 

3100 

0.00 

5,500.00 

0.00- 

0.00 

5,500.00 

. i 

GRAND  TOTALS 

0.00 

147,000.00 

O.OOJ 

0.00 

147,000.001 

Financials 


AH  of  the  Tele-Medicine  funds  were  spent  or  obligated  as  outlined  in  Expenditure  Section 
by  the  end  of  4th  quarter  00.  Funds  were  transferred  to  the  Oak  Ridge  Institute  for 
Science  and  Engineering  (ORISE)  and  the  USACHPPM  Statistician  contract  to  cover  the 
staffing  and  related  project  expenses.  Funds  to  cover  travel  expenses  (noted  above)  were 
transferred  to  ORISE  in  the  4th  quarter  to  cover  ORISE  participant  travel  to  complete 
the  data  collection  phase. 


Final  Results 


In  addition  to  the  three  original  goals  identified  for  the  study  assessing  the  amount  of 
agreement  between  the  on-  site  technicians  and  on-site  and  off-site  ergonomists,  the 
amount  and  type  of  error  was  also  assessed  in  the  analysis.  Two  types  of  errors  were 
identified  in  the  analysis:  1.  Technician  judgement  error.  These  errors  were  identified 
based  on  the  comparison  between  the  technician  and  the  on-site  ergonomist.  Using  the 
on-site  ergonomist  as  the  ‘gold  standard’,  both  ‘false  negative’  (technician  decision  that  it 
was  not  necessary  to  use  a  particular  tool  when  the  on-site  ergonomist  decided  to  use  the 
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instrument)  and  ‘false  positive’  (technician  decision  to  use  a  particular  tool  when  the 
on-site  ergonomist  did  not  use  the  instrument)  errors  were  identified.  2.  Technician 
measurement  process  error.  These  errors  were  identified  by  incomplete,  inaccurate  or 
inappropriate  data  for  each  of  the  tools.  The  results  of  the  agreement  and  error 
assessments  for  each  of  the  assessment  tools  are  presented  below. 

Borg  Scale  The  Borg  Scale  is  a  15-point  scale  that  records  the  soldier’s  self-assessment  of 
physical  exertion  required  by  the  task.  The  soldier  was  asked  to  rate  his/her  level  of 
exertion  during  the  task  on  the  Borg  Scale  when  the  task  was  completed,  a.  Agreement.  A 
total  of  97  tasks  had  valid  Borg  scale  scores  recorded  by  both  the  on  site  ergonomists  and 
the  technicians.  Of  these  97  tasks,  65  (67%)  were  recorded  the  same  by  the  on  site 
ergonomists  and  the  technicians.  The  technician  recorded  16  (16.5%)  scores  of  the  tasks 
higher  than  the  on  site  ergonomist  and  recorded  16  (16.5%)  of  the  tasks  lower  than  the 
on  site  ergonomist.  The  average  Borg  scale  score  for  the  on  site  ergonomist  was  10.5  +/- 
3.2  and  11.0  +/-  3.9  for  the  technician.  The  mean  difference  in  Borg  scale  ratings  from  the 
technician  to  the  on  site  ergonomist  was  —0.4  +/-  3.1.  No  statistically  significant  difference 
in  the  Borg  scale  rating  was  observed  between  the  on-site  ergonomists  and  the 
technicians  using  a  Wilcoxon  Signed  Rank  test.  Agreement  between  the  ergonomists  and 
technicians  was  measured  with  the  weighted  Kappa  statistic,  0.65  (0.48, 0.81)  and  tested 
by  the  marginal  homogeneity  test.  Good  agreement  was  found  and  no  significant 
difference  was  observed  between  the  ergonomist  and  technician  Borg  scale  ratings,  b. 
Error.  A  total  of  108  tasks  had  correct  values  from  the  on  site  ergonomist.  Of  those  108, 

11  (10.2%)  had  an  incorrect  or  no  value  given  by  the  on  site  technician. 

Level  1  Guide  The  Level  1  Guide  is  an  assessment  tool  which  requires  the  classification  of 
the  type  of  task  and  the  recording  of  observations  and  frequency  of  specific  risk  factors 
and  postures  by  body  region. 

Overall  Level  1  Guide:  a.  Agreement.  A  total  of  102  tasks  were  rated  with  valid  scores  by 
both  the  on-site  ergonomists  and  the  technicians.  Of  these  102  tasks,  44  (43.1%)  were 
rated  the  same  by  both  the  ergonomists  and  the  technicians.  The  technicians  rated  30 
(29.4%)of  the  tasks  higher  than  the  on  site  ergonomist  and  28  (27.4%)  of  the  tasks  lower 
than  the  on  site  ergonomist.  The  average  Level  1  Rating  was  1.5  +!- 1.3  for  the  on  site 
ergonomist  and  1.4  -i-/- 1.2  for  the  technician.  The  mean  difference  in  matched  pairs  of 
Level  1  ratings  was  0.02  -e/- 1.49.  No  statistically  significant  difference  in  the  Level  1 
Rating  was  observed  between  the  on-site  ergonomists  and  the  technicians  using  a 
Wilcoxon  Signed  Rank  test.  Agreement  between  the  ergonomists  and  technicians  was 
measured  with  the  weighted  Kappa  statistic,  0.30  (0.13,  0.48)  and  tested  by  the  marginal 
homogeneity  test.  Fair  to  good  agreement  was  found  and  no  significant  difference  was 
observed  between  the  ergonomists’  and  technicians’  Level  1  Ratings,  b.  Error.  A  total  of 
122  tasks  were  given  valid  scores  by  the  onsite  ergonomist.  The  technician  made  a 
judgemental  error  on  31  (25.4%)  of  the  122  tasks  and  a  measurement  error  on  20 
(16.4%)  of  the  tasks. 

Body  Region:  a.  Agreement.  A  total  of  102  tasks  were  given  valid  body  region  scores  by 
both  the  on-site  ergonomists  and  technicians.  Of  these  102  tasks,  43  (42.2%)  were  given 
the  same  body  region  score  by  both  the  ergonomists  and  technicians.  The  technicians 
scored  59  (57.8%)  of  the  tasks  with  different  body  regions  than  the  on-site  ergonomist. 
Agreement  between  the  ergonomists  and  technicians  was  measured  with  a  Kappa 
statistic,  0.22  (0.10, 0.34)  and  tested  by  the  marginal  homogeneity  test.  Poor  agreement 
was  found,  but  no  significant  difference  was  observed  between  the  ergonomists’  and 
technicians’  body  region  scoring,  b.  Error.  Of  the  125  tasks,  the  on-site  ergonomist 


recorded  data  for  122  tasks.  A  judgment  error  was  made  by  the  technician  on  31  (25.4%) 
of  the  122  validly  scored  tasks  and  a  measurement  error  was  made  on  20  (16.4%). 

Identical  error  findings  (above)  were  observed  for  each  body  region  rated  separately, 
therefore  the  error  results  are  not  presented  in  the  body  regions  sections. 

Shoulder/Neck  Rating:  Agreement.  A  total  of  101  tasks  were  rated  with  valid  scores  by 
both  the  on  site  ergonomists  and  technicians.  Of  these  101  tasks,  26  (25.7%)  were  rated 
the  same  by  the  on  site  ergonomists  technicians.  The  technician  rated  38  (37.6%)  of  the 
tasks  higher  and  37  (36.6%)  of  the  tasks  lower  than  the  on  site  ergonomist.  The  median 
(25th,  75th  percentiles)  Shoulder  /  Neck  rating  for  the  on  site  ergonomist  was  2  (-2, 5)  and 
1  (-2, 5)  for  the  technician  (median  values  are  reported  because  they  are  not  affected  by 
the  negative  coding  scheme).  No  statistically  significant  difference  in  the  Shoulder  /  Neck 
rating  was  observed  between  the  on-site  ergonomists  and  technicians  using  a  Wilcoxon 
Signed  Rank  test.  Agreement  between  the  ergonomists  and  technicians  was  measured 
and  tested  with  the  weighted  Kappa  statistic,  0.11  (-0.04, 0.28).  Poor  agreement  was 
found  between  the  ergonomists  and  technicians  in  Shoulder  /  Neck  ratings  and  the 
agreement  that  was  observed  is  due  to  purely  random  variation.  HandAVrist/Arm 
Rating:  Agreement.  A  total  of  102  tasks  were  rated  with  valid  scores  by  both  the  on-site 
ergonomists  and  technicians.  Of  these  102  tasks,  28  (27.4%)  were  rated  the  same  by  the 
on  site  ergonomists  and  technicians.  The  technician  rated  36  (35.3%)  of  the  tasks  higher 
and  38  (37.2%)  of  the  tasks  lower  than  the  on  site  ergonomist.  The  median  (25th,  75th 
percentiles)  Hand  /  Wrist  /Arm  Rating  for  the  on  site  ergonomist  was  2  (-2, 5)  and  2  (-2, 
5.25)  for  the  technician  (median  values  are  reported  because  they  are  not  affected  by  the 
negative  coding  scheme).  No  statistically  significant  difference  in  the  Hand  /  Wrist  /Arm 
Rating  was  observed  between  the  on-site  ergonomists  and  technicians  using  a  Wilcoxon 
Signed  Rank  test.  Agreement  between  the  ergonomists  and  technicians  was  measured 
and  tested  with  the  weighted  Kappa  statistic,  0.13  (-0.03, 0.30).  Poor  agreement  was 
found  between  the  ergonomists  and  technicians  in  Hand  /  Wrist  /Arm  Rating  and  the 
agreement  that  was  observed  is  due  to  purely  random  variation.  Back/Torso  Rating: 
Agreement.  A  total  of  102  tasks  were  rated  with  valid  scores  by  both  the  on  site 
ergonomists  and  technicians.  Of  these  102  tasks,  24  (23.5%)  were  rated  the  same  by  the 
on  site  ergonomists  and  technicians.  The  technicians  rated  35  (34.3%)  of  the  tasks  higher 
and  43  (42.2%)  of  the  tasks  lower  than  the  on  site  ergonomist.  The  median  (25th,  75th 
percentiles)  Back  /  Torso  Rating  for  the  on  site  ergonomist  was  2  (-2,  6)  and  1  (-2,  5)  for 
the  technicians  (median  values  are  reported  because  they  are  not  affected  by  the  negative 
coding  scheme).  No  statistically  significant  difference  in  the  Back  /  Torso  Rating  was 
observed  between  the  on-site  ergonomists  and  technicians  using  a  Wilcoxon  Signed  Rank 
test.  Agreement  between  the  ergonomists  and  technicians  was  measured  and  tested  with 
the  weighted  Kappa  statistic,  0.44  (0.28,  0.60).  Good  agreement  was  found  between  the 
ergonomists  in  Back  /  Torso  Rating,  which  was  also  significantly  different  from  0.0  or  no 
agreement,  p<0.01.  Legs/Feet  Rating:  Agreement.  A  total  of  102  tasks  were  rated  with 
valid  scores  by  both  the  on  site  ergonomists  and  technicians.  Of  these  102  tasks,  32 
(31.4%)  were  rated  the  same  by  the  on  site  ergonomists  and  technicians.  The  technician 
rated  36  (35.3%)  of  the  tasks  higher  and  34  (33.3%)  of  the  tasks  lower  than  the  on  site 
ergonomist.  The  median  (25th,  75th  percentiles)  Legs/Feet  Rating  for  the  on  site 
ergonomist  was  0  (-2, 3)  and  1.5  (-2, 2)  for  the  technician  (median  values  are  reported 
because  they  are  not  affected  by  the  negative  coding  scheme).  No  statistically  significant 
difference  in  the  Legs/Feet  Rating  was  observed  between  the  on  site  ergonomists  and 
technicians  using  a  Wilcoxon  Signed  Rank  test.  Agreement  between  the  ergonomists  and 
technicians  was  measured  and  tested  with  the  weighted  Kappa  statistic,  0.28  (0.12,  0.44). 
Fair  agreement  was  found  between  the  ergonomists  and  technicians  in  Legs/Feet  Rating, 
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which  was  also  significantly  different  from  0.0  or  no  agreement,  p<0.01.  Head/Eyes 
Rating:  Agreement.  A  total  of  100  tasks  were  rated  with  valid  scores  by  both  the  on  site 
ergonomists  and  technicians.  Of  these  100  tasks,  62  (62.0%)  were  rated  the  same  by  the 
on  site  ergonomists  and  technicians.  The  technicians  rated  25  (25.0%)  of  the  tasks  higher 
and  13  (13.0%)  of  the  tasks  lower  than  the  on  site  ergonomist.  The  median  (25th,  75th 
percentiles)  Head/Eyes  Rating  for  the  on  site  ergonomist  was  0  (-2, 0)  and  0  (-2, 0)  for  the 
technician  (median  values  are  reported  because  they  are  not  affected  by  the  negative 
coding  scheme).  No  statistically  significant  difference  in  the  Head/Eyes  Rating  was 
observed  between  the  on  and  offsite  ergonomists  using  a  Wilcoxon  Signed  Rank  test. 
Agreement  between  the  ergonomists  and  technicians  was  measured  and  tested  with  the 
weighted  Kappa  statistic,  0.21  (0.00, 0.42).  Poor  agreement  was  found  between  the 
ergonomists  and  technicians  in  Head/Eyes  Rating,  which  was  also  significantly  different 
from  0.0  or  no  agreement,  p<0.05.  Environmental  Rating:  Agreement.  A  total  of  101 
tasks  were  rated  with  valid  scores  by  both  the  on  site  ergonomists  and  technicians.  Of 
these  101  tasks,  40  (39.6%)  were  rated  the  same  by  the  on  site  ergonomists  and 
technicians.  The  technicians  rated  34  (33.7%)  of  the  tasks  higher  and  27  (26.7  %)  of  the 
tasks  lower  than  the  on  site  ergonomist.  The  median  (25th,  75th  percentiles) 
Environmental  Rating  for  the  on  site  ergonomist  was  0  (-2, 1)  and  0  (-2, 1)  for  the 
technician  (median  values  are  reported  because  they  are  not  affected  by  the  negative 
coding  scheme).  No  statistically  significant  difference  in  the  Environmental  Rating  was 
observed  between  the  on  site  ergonomists  and  technicians  using  a  Wilcoxon  Signed  Rank 
test.  Agreement  between  the  ergonomists  and  technicians  was  measured  and  tested  with 
the  weighted  Kappa  statistic,  0.12  (-0.11, 0.34).  Poor  agreement  was  found  between  the 
ergonomists  and  technicians  in  Environmental  Rating  and  the  agreement  that  was 
observed  is  due  to  purely  random  variation.  NIOSH  Revised  Lifting  Equation  This 
assessment  tool  is  based  on  measurements  of  six  lift  characteristics.  These  measurements 
are  entered  into  a  calculation  which  results  in  a  recommended  weight  limit  and  a  risk 
index  for  the  lift.  The  on-site  ergonomist  and  technicians  were  required  to  identify  any 
lifts  requiring  assessment  and  measure  the  lift  up  to  a  total  of  five  lifts  per  task. 
Comparison  of  the  on  site  ergonomist  to  the  technicians  is  most  valid  for  the  NIOSH  Lift 
rating  for  lifts  1  and  2  because  the  majority  of  the  lifts  3-5  were  considered  not  necessary 
to  be  rated.  As  there  were  minimal  differences  between  lifts  3, 4  and  5,  only  lift  3  is 
presented  here. 

NIOSH  Lift  1  Rating,  a.  Agreement.  Of  the  125  tasks  that  were  rated  by  both  the  on-site 
ergonomists  and  technicians,  78  (62.4%)  had  a  valid  score  from  both  ergonomists  and 
technicians.  Of  those  78  tasks,  68  (87.2%)  were  rated  the  same  by  both  ergonomists  and 
technicians.  The  technician  rated  5  (6.4%)  as  not  necessary  for  rating  when  the  on  site 
ergonomist  rated  them  and  5  (6.4%)  with  a  rating  when  the  on  site  ergonomist  rated 
them  as  not  necessary  for  rating.  Agreement  between  the  ergonomists  and  technicians 
was  measured  with  the  Kappa  statistic,  0.65  (0.45, 0.85)  and  tested  by  the  marginal 
homogeneity  test.  Good  agreement  was  found  and  no  significant  difference  was  observed 
between  the  ergonomists’  and  technicians’  NIOSH  Lift  1  ratings,  b.  Error.  Of  the  125 
tasks,  108  were  completed  with  valid  scores  by  the  ergonomist.  Of  those  108,  the 
technician  made  judgemental  errors  on  10  (9.2%)  and  measurement  errors  on  30 
(27.8%). 

NIOSH  Lift  2  Rating,  a.  Agreement.  Of  the  125  tasks  that  were  rated  by  both  the  on-site 
ergonomists  and  technicians,  102  (81.6%)  had  a  valid  score  from  both  ergonomists  and 
technicians.  Of  those  102  tasks,  92  (90.2%)  were  rated  the  same  by  the  on-  site 
ergonomists  and  technicians.  The  technicians  rated  9  (8.8%)  as  not  necessary  for  rating 
when  the  on  site  ergonomist  rated  them  and  1  (1.0%)  with  a  rating  when  the  on  site 
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ergonomist  rated  them  as  not  necessary  for  rating.  Agreement  between  the  ergonomists 
and  technicians  was  measured  with  the  Kappa  statistic,  0.56  (0.33, 0.80)  and  tested  by  tbe 
marginal  homogeneity  test.  The  Kappa  statistic  indicated  good  agreement,  but  the 
marginal  homogeneity  test  was  significant  indicating  a  significant  difference  between 
ergonomists  for  the  NIOSH  Lift  2  ratings,  b.  Error.  Of  the  125  tasks,  116  were  completed 
with  valid  scores  by  the  ergonomist.  Of  those  116,  the  technician  made  judgemental 
errors  on  10  (8.6%)  and  measurement  errors  on  14  (12.1%). 

NIOSH  Lift  3  Rating,  a.  Agreement.  Of  the  125  tasks  that  were  rated  by  both  on-site 
ergonomists  and  technicians,  107  (85.6%)  had  a  valid  score  from  both  ergonomists  and 
technicians.  Of  those  107  tasks,  101  (94.4%)  were  rated  the  same  by  both  ergonomists 
and  technicians.  The  technicians  rated  3  (2.8%)  as  not  necessary  for  rating  when  the  on 
site  ergonomist  rated  them  and  3  (2.8%)  with  a  rating  when  the  on  site  ergonomist  rated 
them  as  not  necessary  for  rating.  Agreement  between  the  ergonomists  and  technicians 
was  measured  with  the  Kappa  statistic,  0.37  (-0.02, 0.77)  and  tested  by  the  marginal 
homogeneity  test.  Fair  agreement  was  found  and  no  significant  difference  was  observed 
between  the  ergonomists’  and  technicians’  NIOSH  Lift  3  ratings,  b.  Error.  Of  the  125 
tasks,  the  ergonomist  had  valid  scores  on  119  tasks.  The  technician  made  judgemental 
errors  on  6  (5%)  and  measurement  errors  on  12  (10.3%). 

Rodgers  Methodology:  This  exposure  risk  assessment  tool  evaluates  continuous  effort 
and  efforts/minute  and  results  in  a  priority  for  change  rating,  a.  Agreement.  The  specific 
body  regions  are  discussed  below.  Overall,  there  was  poor  agreement  between  the  on-  site 
ergonomists  and  the  technicians  in  their  rating  of  body  regions  using  Rodgers 
methodology.  Ratings  were  given  by  each  ergonomist  and  technician,  scored  as 
Moderate,  High  or  Very  High.  Some  ratings  were  blank  and  others  incorrect.  Therefore, 
only  tasks  with  valid  scores  by  both  the  on-site  ergonomists  and  technicians  were  used.  b. 
Error.  The  number  of  tasks  rated  by  the  on-site  ergonomist  ranged  from  103  to  111  of 
the  total  125  tasks.  Across  all  body  regions,  of  the  tasks  rated  by  the  on-site  ergonomist, 
30.3%  to  38.7%  of  the  tasks  were  not  rated  by  the  technician  (judgement  error). 

Neck/  Shoulder  -  Right:  a.  Agreement.  Of  the  125  tasks  reviewed  by  both  on-site 
ergonomists  and  technicians,  74  (59.2%)  had  valid  ratings  by  both  ergonomists  and 
technicians  for  the  right  neck  /  shoulder.  Of  those  74  valid  ratings,  57  (77.0%)  were  rated 
the  same  by  both  on-site  ergonomists  and  technicians.  The  technicians  rated  12  (16.2%) 
lower  and  5  (6.8%)  higher  than  the  on  site  ergonomist.  Agreement  between  the 
ergonomists  and  technicians  was  measured  with  the  weighted  Kappa  statistic,  0.12  (-0.15, 
0.39)  and  tested  by  the  marginal  homogeneity  test.  Poor  agreement  was  found  between 
the  on-site  ergonomists  and  technicians  and  the  agreement  that  was  observed  is  due  to 
purely  random  variation  (Table  1). 

b.  Error.  The  on  site  ergonomist  had  valid  scores  on  109  tasks.  Of  these,  the  technician 
made  judgement  errors  on  35  (32.1%). 

Neck  /  Shoulder  -  Left:  a.  Agreement.  Of  the  125  tasks  reviewed  by  both  the  on-site 
ergonomists  and  technicians,  76  (60.8%)  had  valid  ratings  by  both  ergonomists  and 
technicians  for  the  left  neck  /  shoulder.  Of  those  76  valid  ratings,  59  (77.6%)  were  rated 
the  same  by  both  ergonomists  and  technicians.  The  technicians  rated  12  (15.8%)  lower 
and  5  (6.6%)  higher  than  the  on  site  ergonomist.  Agreement  between  the  ergonomists 
and  technicians  was  measured  with  the  weighted  Kappa  statistic,  0.18  (-0.07, 0.44)  and 
tested  by  the  marginal  homogeneity  test.  Poor  agreement  was  found  between  the  on  site 
ergonomists  technicians  and  the  agreement  that  was  observed  is  due  to  purely  random 


variation  (Table  2).  b.  Error.  The  on  site  ergonomist  had  valid  scores  on  109  tasks.  Of 
these,  the  technician  made  judgement  errors  on  33  (30.3%). 

Back:  a.  Agreement.  Of  the  125  tasks  revievred  by  both  the  on-  site  ergonomists  and 
technicians,  75  (60%)  had  valid  ratings  by  both  ergonomists  and  technicians  for  the 
back.  Of  those  75  valid  ratings,  51  (68%)  were  rated  the  same  by  both  the  ergonomists 
and  technicians.  The  technicians  rated  16  (21.3%)  lower  and  8  (10.7%)  higher  than  the 
on  site  ergonomist.  Agreement  between  the  ergonomists  and  technicians  was  measured 
with  the  weighted  Kappa  statistic,  0.05  (-0.18,  0.27)  and  tested  by  the  marginal 
homogeneity  test.  Poor  agreement  was  found  between  the  on-site  ergonomists  and 
technicians  and  the  agreement  that  was  observed  is  due  to  purely  random  variation 
(Table  3).  b.  Error.  The  on  site  ergonomist  had  valid  scores  on  109  tasks.  Of  these,  the 
technician  made  judgement  errors  on  34  (31.2%).  Arm  -  Right:  a.  Agreement.  Of  the  125 
tasks  reviewed  by  both  the  on-  site  ergonomists  and  technicians,  69  (55.2%)  had  valid 
ratings  by  both  ergonomists  and  technicians  for  the  right  arm.  Of  those  69  valid  ratings, 
51  (73.9%)  were  rated  the  same  by  both  ergonomists  and  technicians.  The  technicians 
rated  13  (18.8%)  lower  and  5  (7.2%)  higher  than  the  on  site  ergonomist.  Agreement 
between  the  ergonomists  and  technicians  was  measured  with  the  weighted  Kappa 
statistic,  0.11  (-0.11, 0.33)  and  tested  by  the  marginal  homogeneity  test.  Poor  agreement 
was  found  between  the  on-site  ergonomists  and  technicians  and  the  agreement  that  was 
observed  is  due  to  purely  random  variation  (Table  4).  b.  Error.  The  on  site  ergonomist 
had  valid  scores  on  102  tasks.  Of  these,  the  technician  made  judgement  errors  on  33 
(32.4%). 

Arm  -  Left:  a.  Agreement.  Of  the  125  tasks  reviewed  by  both  ergonomists,  68  (54.4%) 
had  valid  ratings  by  both  the  on-  site  ergonomists  and  technicians  for  the  left  arm.  Of 
those  68  valid  ratings,  53  (77.9%)  were  rated  the  same  by  both  ergonomists  and 
technicians.  The  technicians  rated  8  (11.8%)  lower  and  7  (10.3%)  higher  than  the  on  site 
ergonomist.  Agreement  between  the  ergonomists  and  technicians  was  measured  with  the 
weighted  Kappa  statistic,  0.03  (-0.18, 0.23)  and  tested  by  the  marginal  homogeneity  test. 
Poor  agreement  was  found  between  the  on-site  ergonomists  and  technicians  and  the 
agreement  that  was  observed  is  due  to  purely  random  variation  (Table  5).  b.  Error.  The 
on  site  ergonomist  had  valid  scores  on  103  tasks.  Of  these,  the  technician  made 
judgement  errors  on  35  (34.0%). 

Wrist  -  Right:  a.  Agreement.  Of  the  125  tasks  reviewed  by  both  the  on-  site  ergonomists 
and  technicians,  65  (52%)  had  valid  ratings  by  both  ergonomists  and  technicians  for  the 
right  wrist.  Of  those  65  valid  ratings,  39  (60%)  were  rated  the  same  by  both  ergonomists 
and  technicians.  The  technicians  rated  20  (30.8%)  lower  and  6  (9.2%)  higher  than  the  on 
site  ergonomist.  Agreement  between  the  ergonomists  and  technicians  was  measured  with 
the  weighted  Kappa  statistic,  0.13  (-0.08, 0.35)  and  tested  by  the  marginal  homogeneity 
test.  Poor  agreement  was  found  between  the  on  site  ergonomists  and  technicians  and  the 
agreement  that  was  observed  is  due  to  purely  random  variation.  In  addition,  a  significant 
difference  was  observed  between  the  ergonomists  and  technicians  in  the  rating  of  right 
wrist  by  Rodgers  Methodology,  p<0.05  (Table  6).  b.  Error.  The  on  site  ergonomist  had 
valid  scores  on  104  tasks.  Of  these,  the  technician  made  judgement  errors  on  39  (37.5%). 

Wrist  -  Left:  a.  Agreement.  Of  the  125  tasks  reviewed  by  both  the  on-  site  ergonomists 
and  technicians,  67  (53.6%)  had  valid  ratings  by  both  ergonomists  and  technicians  for 
the  left  wrist.  Of  those  67  valid  ratings,  40  (59.7  % )  were  rated  the  same  by  both 
ergonomists  and  technicians.  The  technicians  rated  18  (26.9%)  lower  and  9  (13.4%) 
higher  than  the  on  site  ergonomist.  Agreement  between  the  ergonomists  and  technicians 


was  measured  with  the  weighted  Kappa  statistic,  -0.01  (-0.19, 0.17)  and  tested  by  the 
marginal  homogeneity  test.  Poor  agreement  was  found  between  the  on  site  ergonomists 
and  technicians  and  the  agreement  that  was  observed  is  due  to  purely  random  variation 
(Table  7).  b.  Error.  The  on  site  ergonomist  had  valid  scores  on  104  tasks.  Of  these,  the 
technician  made  judgement  errors  on  37  (35.6%). 


Legs  -  Right:  a.  Agreement.  Of  the  125  tasks  reviewed  by  both  the  on-  site  ergonomists 
and  technicians,  74  (59.2%)  had  valid  ratings  by  both  ergonomists  and  technicians  for 
the  right  leg.  Of  those  74  valid  ratings,  61  (82.4%)  were  rated  the  same  by  both 
ergonomists  and  technicians.  The  technicians  rated  7  (9.5%)  lower  and  6  (8.1%)  higher 
than  the  on  site  ergonomist.  Agreement  between  the  ergonomists  and  technicians  was 
measured  with  the  weighted  Kappa  statistic,  0.02  (-0.18, 0.22)  and  tested  by  the  marginal 
homogeneity  test.  Poor  agreement  was  found  between  the  on-site  ergonomists  and 
technicians  and  the  agreement  that  was  observed  is  due  to  purely  random  variation 
(Table  8).  b.  Error.  The  on  site  ergonomist  had  valid  scores  on  111  tasks.  Of  these,  the 
technician  made  judgement  errors  on  37  (33.3%). 


Legs  -  Left:  a.  Agreement.  Of  the  125  tasks  reviewed  by  both  the  on-  site  ergonomists  and 
technicians,  75  (60%)  had  valid  ratings  by  both  ergonomists  and  technicians  for  the  left 
leg.  Of  those  75  valid  ratings,  62  (82.7%)  were  rated  the  same  by  both  ergonomists  and 
technicians.  The  technicians  rated  6  (8%)  lower  and  7  (9.3%)  higher  than  the  on  site 


ergonomist.  Agreement  between  the  ergonomists  and  technicians  was  measured  with  the 
weighted  Kappa  statistic,  0.13  (-0.12, 0.37)  and  tested  by  the  marginal  homogeneity  test. 
Poor  agreement  was  found  between  the  on  site  ergonomists  and  technicians  and  the 
agreement  that  was  observed  is  due  to  purely  random  variation  (Table  9).  b.  Error.  The 
on  site  ergonomist  had  valid  scores  on  111  tasks.  (Jt  these,  the  technician  made 
judgement  errors  on  36  (32.4%).  Ankles  -  Right:  a.  Agreement.  Of  the  125  tasks  reviewed 
by  both  the  on-  site  ergonomists  and  technicians,  68  (54.4%)  had  valid  ratings  by  both 
ergonomists  and  technicians  for  the  right  ankles.  Of  those  68  valid  ratings,  54  (79.4%) 
were  rated  the  same  by  both  ergonomists  and  technicians.  The  technicians  rated  8 
(11.8%)  lower  and  6  (8.8%)  higher  than  the  on  site  ergonomist.  Agreement  between  the 
ergonomists  and  technicians  was  measured  with  the  weighted  Kappa  statistic,  0.07  (-0.15, 
0.30)  and  tested  by  the  marginal  homogeneity  test.  Poor  agreement  was  found  between 
the  on  site  ergonomists  and  technicians  and  the  agreement  that  was  observed  is  due  to 
purely  random  variation  (Table  10).  b.  Error.  The  on  site  ergonomist  had  valid  scores  on 
111  tasks.  Of  these,  the  technician  made  judgement  errors  on  43  (38.7 %). 


Ankles  -  Left:  a.  Agreement.  Of  the  125  tasks  reviewed  by  both  the  on-  site  ergonomists 
and  technicians,  68  (54.4%)  had  valid  ratings  by  both  ergonomists  and  technicians  for 
the  left  ankles.  Of  those  68  valid  ratings,  56  (82.4%)  were  rated  the  same  by  both 
ergonomists  and  technicians.  The  technicians  rated  6  (8.8%)  lower  and  6  (8.8%)  higher 
than  the  on  site  ergonomist.  Agreement  between  the  ergonomists  and  technicians  was 
measured  with  the  weighted  Kappa  statistic,  0.13  (-0.12, 0.38)  and  tested  by  the  marginal 
homogeneity  test.  Poor  agreement  was  found  between  the  on  site  ergonomists  and 
technicians  and  the  agreement  that  was  observed  is  due  to  purely  random  variation 
(Table  11).  b.  Error.  The  on  site  ergonomist  had  valid  scores  on  111  tasks.  Of  these,  the 
technician  made  judgement  errors  on  43  (38.7%). 

Ovako  Working  Posture  Analysis  System  (OWAS).  The  OWAS  assesses  the  presence  of 
high  risk  static  and  dynamic  postures  of  the  trunk,  upper  limbs,  lower  limbs,  head  and 
neck,  and  the  load  of  stress  experienced  by  tbe  body.  This  method  provides  a  measure  of 
the  frequency  of  high-risk  static  and  dynamic  postures  in  each  of  the  above-  specified 
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anatomic  regions.  Load  is  assessed  by  classifying  the  weight  lifted  or  strength  required 
into  three  categories:  <10  kg,  10-20  kg,  >  20  kg.  By  summing  observations,  the  OWAS 
provides  the  percentage  of  time  in  selected  postures.  The  OWAS  evaluation  is  based  on  a 
time-sampling  of  a  recorded  videotape.  The  off-site  ergonomist’s  ability  to  assess  the  task 
using  the  OWAS  is  dependant  on  the  technician’s  videotaping  skills,  focus  and 
concentration.  The  total  of  the  OWAS  Categories  1-4  was  calculated  and  a  percent  of  the 
total  score  was  used  for  each  OWAS  category.  The  on  site  and  off  site  ergonomists  were 
compared  using  the  percentages  for  each  category  with  a  paired  t-test  (Table  12). 

Rapid  Upper  Limb  Assessment  (RULA):  The  RULA  assessment  method  measures 
working  posture,  muscle  use  and  force  required  to  perform  job  tasks.  It  produces  a 
summary  score  representing  all  measured  variables  and  a  priority  for  change  score.  The 
RULA  is  also  based  on  a  time-sampling  of  a  recorded  videotape.  The  off-site 
ergonomists’  assessment  using  RULA  is  dependent  on  the  technician’s  videotaping  skills. 
The  total  of  the  RULA  Categories  1-4  was  calculated  and  a  percent  of  the  total  score  was 
used  for  each  RULA  category.  The  on  site  and  off  site  ergonomists  were  compared  using 
the  percentages  for  each  category  with  a  paired  t-test  (Table  13). 

Technician  Usability  Assessment.  After  the  task  data  had  been  collected,  tbe  technicians 
completed  a  questionnaire-based  rating  of  each  assessment  tool  to  determine  the  usability 
of  the  tools.  The  Borg  Scale  was  rated  the  most  favorable  for  every  question.  The  mean 
responses  from  the  assessment  tool  rating  questionnaire  are  provided  in  Table  14. 

Because  every  subject  scored  each  method  for  every  question,  a  repeated  measures 
analysis  of  variance  was  used  to  compare  the  methods.  If  a  significant  method  effect  was 
observed,  then  a  Tukey’s  multiple  comparisons  test  was  used  to  make  the  pair-wise 
comparisons.  Statistical  significance  was  defined  as  p<0.05.  Questions  1, 2,  4  and  6 
showed  significant  method  effects,  p<0.05.  While  the  mean  method  scores  for  Question  10 
appeared  to  be  different,  they  failed  to  reach  statistical  significance,  p=0.072.  No  other 
questions  showed  significant  differences  between  tbe  methods  used.  Question  1:  How 
easy  was  it  to  collect  the  information?  The  mean  score  for  NIOSH  Lift  Equation  was 
significantly  different  than  all  other  method’s  mean  scores,  p<0.05.  The  mean  score  for 
the  Rodgers  Methodology  was  significantly  different  from  the  mean  score  for  the  Borg 
scale  method,  p<0.05  (Figure  1).  Question  2:  If  you  had  to  collect  the  same  information  in 
the  field,  how  easy  do  you  think  it  would  be?  The  mean  score  for  the  NIOSH  Lift 
Equation  was  significantly  different  from  the  mean  score  for  the  Borg  scale  method, 
p<0.05  (Figure  2).  Question  4:  How  much  time  did  it  take  to  collect  this  information?  The 
mean  score  for  the  Video  and  NIOSH  Lift  Equation  methods  was  significantly  different 
from  the  mean  score  for  the  Borg  scale  method,  p<0.05  (Figure  3).  Question  6:  How  much 
time  will  it  take  to  collect  this  information  in  a  deployed  environment?  The  mean  score 
for  the  Borg  Scale  method  was  significantly  different  than  all  other  method’s  mean 
scores,  p<0.05  (Figure  4).  Question  10:  How  complete  were  the  instructions  for  the 
assessment  method?  Although  differences  in  the  mean  scores  did  not  reach  statistical 
significance,  responses  to  this  question  are  relevant  for  practical  purposes  (Figure  5). 

Conclusions  One  of  the  primary  goals  of  this  research  was  the  identification  of  an 
ergonomic  assessment  tool  that  could  easily  be  used  by  unit  health  and  safety  personnel 
in  the  garrison,  field,  and  deployed  environments.  The  assessment  tools  were  evaluated  in 
three  general  areas  —  agreement  between  the  on-site  ergonomist  and  the  technician  and 
off-  site  ergonomist;  error  potential;  and  usability.  Good  agreement  was  seen  with  the 
Borg  scale.  Fair  to  good  agreement  was  seen  with  the  Level  1  Guides;  however,  this 
agreement  was  not  seen  in  the  individual  scales  of  the  instrument.  Good  to  fair  agreement 
was  seen  with  lifts  1-2  for  the  NIOSH  Lifting  Equation.  The  Rodgers  methodology 
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showed  poor  agreement  in  general  and  in  the  individual  body  categories.  Both  the 
OWAS  and  RULA  were  dependent  on  the  quality  of  the  videotape  and  showed 
significant  differences  between  the  on-  and  off-site  ergonomists.  The  Borg  Scale  ranked 
low  in  judgement  and  measurement  errors.  The  Level  1  Guides  had  significantly  more 
errors,  ranking  medium  in  both  judgement  and  measurement  errors.  The  NIOSH  Lifting 
Equation  had  fewer  judgement  errors  but  significantly  more  measurement  errors.  The 
Rodgers  Methodology  had  some  measurement  errors,  but  significantly  more  judgement 
errors.  The  technicians  involved  in  this  study  rated  the  Borg  Scale  as  the  easiest  method 
to  use,  as  well  as  the  method  which  took  the  least  amount  of  time  to  collect.  However,  the 
perceived  ease  of  use  of  the  Borg  Scale  was  not  significantly  greater  than  the  Level  1 
Assessment,  Rodgers,  or  the  Video  methods. 


Projected  Costs 


The  next  step  is  to  revise  the  technician’s  assessment  tool  package.  Only  the  Borg  scale 
and  the  videotaping  will  be  included  in  the  package  and  the  package  will  be  re-  formatted 
to  provide  a  decision  tree,  clearer  and  more  structured  instructions  and  a  training 
session.  This  package  will  be  posted  on  the  Ergonomics  web  page  for  easier  access.  In 
addition,  modifications  to  the  91S  curriculum  will  be  proposed  and  developed  to  provide 
a  stronger  background  in  this  type  of  data  collection.  This  revised  methodology  will  be 
presented  and  tested  for  use  in  deployed  environments. 

At  a  local  level,  the  projected  costs  are  minimal.  The  web-  accessible  tools  and 
self-training  will  be  available  at  no  cost.  The  only  additional  cost  will  be  the  manpower 
time  required  to  collect  this  data  and  the  cost  of  a  video  camera;  however,  those  costs  will 
be  offset  by  the  reduced  time  to  conduct  evaluations  due  to  the  greater  structure 
provided  and  the  cost-savings  in  the  expert  ergonomist  consultations.  In  addition,  any 
local  costs  will  also  be  offset  by  the  reduced  injury  risk  to  both  military  and  civilian 
personnel  and  the  more  timely  response  to  problem  areas. 

On  an  AMEDD-wide  level,  the  projected  costs  are  also  minimal.  The  modifications  to  the 
assessment  tool  package  will  require  some  manpower  time;  however,  as  was  noted 
previously,  those  costs  will  be  easily  offset  by  eliminating  the  travel  time  and  costs  and 
response  time  delays  as  well  as  increasing  access  to  expert  consultation.  The  proposed 
curriculum  changes  will  also  require  some  manpower  time  investment,  but  continuous 
updating  and  quality  improvement  of  the  curriculum  is  expected  and  accounted  for  in 
education. 
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Comments 


It  is  clear  from  this  study  that  the  technicians  require  more  structure  in  a  user-friendly 
format  to  reduce  the  judgement  and  measurement  errors.  The  technicians  also  require 
hands  on  training  in  the  measurement  techniques.  It  is  important  to  note  that  the 
technicians’  self-reported  comfort  level  with  the  assessment  tools  and  videotaping 
increased  dramatically  with  even  a  small  amount  of  practice.  The  Borg  scale  was  clearly 
the  best  assessment  tool  to  include  in  the  package.  The  Level  1  Guide  has  potential; 
however,  more  training  is  required  to  use  this  tool  and  the  length  of  time  required  to 
complete  the  assessment  may  be  prohibitive  in  some  situations.  The  most  critical 
information  source  for  the  off-site  ergonomist  was  the  videotape  recording  of  the  task. 
This  was  also  the  root  cause  for  the  lack  of  agreement  between  the  on-site  and  off-site 
ergonomists.  Clearer,  more  structured  and  user  friendly  instructions  and  practice  are 
required  before  the  technician-collected  data  will  be  meaningful. 

Work-related  musculoskeletal  disorders  are  the  greatest  readiness  concern  facing  the 
Department  of  Defense.  These  injuries  represent  the  greatest  source  of  disease  non-  battle 
injuries  and  are  the  primary  reason  for  the  majority  of  physical  limitation  profiles. 
Ergonomic  redesigns  of  tasks,  tools  and  equipment  based  on  accurate  ergonomic 
assessments  can  reduce  injuries  and  re-injuries,  reduce  lost  work  time  and  limited  duty 
days,  facilitate  return  to  work  and  improve  overall  unit  readiness.  Results  of  this  study 
will  have  far-reaching  effects  by  providing  the  necessary  assessment  tools  and  processes 
to  conduct  accurate  and  timely  ergonomic  assessments  in  all  military  environments  and 
locations. 
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Table  1 
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Table  2 
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Table  3 
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Table  4 
Arm  -  Right: 
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Table  5 
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Table  6 
Wrist  -  Right: 
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Table  7 
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Table  8 
Legs  -  Right: 
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Table  9 
Legs  -  Left: 
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Table  10 
Ankles  -  Right: 
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Table  11 
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Table  12 

Mean  ±  SEM  OWAS  by  category  (n=125) 


OWAS  1  (%) 

OWAS  2  (%) 

OWAS  3  (%) 

OWAS  4  (%) 

68.0  +  2.4 

21.8  +  1.9 

6.0+ 1.2 

4.2 +  1.1 

Off  Site  * 

51.9  +  2.7 

37.4  +  2.5 

8.6+  1.4 

2.2  +  0.9 

*  Significantly  different  than  the  on  site  ergonomist’s  reading  for  OWAS  1, 


OWAS  2  and  OWAS  4,  p<0.05. 


Table  13 

Mean  +  SEM  RULA  by  category  (n=125) 


RULA  1  (%) 

RULA  2  (%) 

RULA  3  (%) 

RULA  4  (%) 

On  Site 

44.1  ±2.2 

6.7  ±1.2 

Off  Site  * 

39.2  +  2.4 

43.9  +  2.3 

13.0+1.8 

3.9+  1.1 

*  Significantly  different  than  the  on  site  ergonomist’s  reading  for  RULA  1, 


RULA  3  and  RULA  4,  p<0.05. 


Table  14 

Questionnaire  Responses  -  Mean  Values. 


Question  Video  NIOSH  Borg  Rodgers  Level 

Method  Lift  Scale  Methodology  1 
_ Equation _ Guide 

1.  How  easy  was  it  to  2.33  3.78  1.44  2.56  2.22 

collect  the 

information? _ 

2.  If  you  had  to  collect  2.56  3.44  2.22  3.22  2.56 

the  same  information 

in  the  field,  how  easy 
do  you  think  it  would 

be? _ 

3.  If  you  had  to  collect  2.67  2.89  2.33  3  2.44 

the  same  information 

in  a  deployed 
environment,  how  easy 
do  you  think  it  would 

be? _ 

A/  How  much  time  4.22  3.89  2.33  3.22  3.11 

did  it  take  to  collect 

this  information? _ 

5.  ■  How  much  time  3.78  3.78  2.78  3.44  3.44 

will  it  take  to  collect 

this  information  in  the 

field? _ 

6. '  Howmuchtime  3.78  3.78  2.33  3.11  3.33 

will  it  take  to  collect 

this  information  in  a 

deployed  environment? _ 

7.  How  “user-  2.44  2.22  1.56  2.44  2.44 

friendly”  was  this 

assessment  method? _ 

8.  How  helpful  were  2.33  2.11  1.67  2.33  2.56 

the  instmctions  for  this 

assessment  method? _ 

9.  How  clear  were  the  2.22  2.11  1.67  2.44  2.56 

instructions  for  this 

assessment  method? _ 

10.  How  complete  2.11  2.11  1.56  2.33  2.56 

were  the  instmctions 

for  this  assessment 

method? _ 

11.  Overall,  how  2.22  2.78  2  2.44  2.33 

accurate  was  the  data  _ 


you  collected  with  this 
assessment  method? 

12.*  How  many  task 
assessments  did  you 
perform  with  this 
assessment  method 
before  you  felt  the  data 
collected  was  accurate? 
Please  provide  a 
specific  number  of 
tasks. 

6.44 

8.78 

4.11 

7.33 

7 

13.*  How  many  task 
assessments  did  you 
perform  before  you  felt 
comfortable  with  this 
assessment  method? 
Please  provide  a 
specific  number  of 
tasks. 

4.89 

9.11 

4.44 

7.78 

7.11 

■  These  questions  were  on  a  Likert  scale.  They  did  not  ask  for  a  specific  amount 


of  time. 

*  These  questions  were  not  on  the  Likert  scale.  They  asked  for  a  specific  number 
of  tasks. 


Figure  1  :  How  easy  was  it  to  collect  the  information? 


'^Boro  ”Levpl1  °NIOSH  Rodgers  °  Video 


MurntM  r  oT  R^sportaea  Num^  r  qt  Fte3poriee& 


Figure  2  :  If  you  had  to  collect  the  same  information  in  the  field,  how 
easy  do  you  think  it  would  be? 
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Figure  3  :  How  much  time  did  it  take  to  collect  this  information? 
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Figure  4  :  How  much  time  will  it  take  to  collect  this  information  in  a 
deployed  environment? 
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Figure  5  :  How  complete  were  the  instructions  for  this  assessment 

method? 
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